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SUMMARY 


This  interim  report  describes  research  advances  on  stochastic  and 
adaptive  systems  by  faculty,  research  staff,  and  students  of  the  M.I.T. 
Laboratory  for  Information  and  Decision  Systems  with  support  provided  by 
the  United  States  Air  Force  Office  of  Scientific  Research  tinder  Grant 
AFOSR  77-3281C.  The  principal  co-investigators  were  Professors  Michael 
Athans  and  San joy  K.  Mitter.  The  Grant  Monitor  was  Dr.  Joseph  Bram. 

The  time  period  covered  by  this  report  is  February  1,  1980  to  January  31, 
1981. 

Substantial  progress  is  reported  in  the  areas  of  nonlinear  filtering 
theory,  stochastic  control,  and  adaptive  control  systems. 
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1.  INTRODUCTION 

The  research  we  have  conducted  over  the  past  several  years  and  in 
particular  during  the  period  February  1,  1980  to  January  31,  1981  has  been 
concerned  with  fundamental  aspects  of  controlling  linear  and  non-linear 
stochastic  systems  in  the  presence  of  measurement  and  parameter  uncertainties. 
In  case  the  uncertainties  reside  in  the  state  description  of  the  physical 
system  and  measurements  then  we  refer  to  the  control  problem  as  a  stochastic 
control  problem.  If  in  addition  there  are  parameter  uncertainties,  then  the 
problem  is  referred  to  as  an  adaptive  control  problem.  This  is  because 
in  addition  to  state  estimation  some  form  of  parameter  identification 
scheme  will  be  needed  and  almost  always  the  control  and  estimation- identifi¬ 
cation  functions  will  interact  in  a  non-trivial  nonlinear  and  time-varying 
way. 

A  subproblem  of  the  stochastic  and  adaptive  control  problem  is  the  state 
estimation  problem.  Suppose  for  a  moment  we  make  the  assumption  that  there 
are  no  parameter  uncertainties  present  in  the  dynamical  description  of  the 
state  of  the  system.  Then  all  the  probabilistic  information  that  one  can 
extract  about  the  "state"  of  the  system  on  the  basis  of  noisy  measurements 
of  the  state  is  contained  in  the  conditional  probability  density  of  the 
state  given  the  observations.  Indeed  this  is  the  probabilistic  state  of 
the  joint  physical-measurement  system.  The  recursive  computation  of  the 
probabilistic  state  is  the  state  estimation  problem.  If  this  could  be  done, 
then  one  could  look  for  the  best  controller  as  a  function  of  this  probabilistic 
state,  best  being  judged  in  terms  of  a  suitable  performance  criterion. 
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We  believe  that  we  have  only  scratched  the  surface  in  understanding  some 
of  these  fundamental  issues  in  modelling,  estimation,  and  control  of  linear 
systems  with  multiplicative  uncertain  parameters  and,  more,  generally  non¬ 
linear  systems.  Our  proposed  research  deals  with  those  issues  that  we 
feel  are  both  fundamental  and  relevant  from  an  engineering  point  of  view. 

We  would  like  to  mention  that  many  of  the  theoretical  questions  we  are 
posing  have  been  motivated  by  problems  which  are  of  importance  to  the  U.S. 
Air  Force.  Modelling,  estimation,  and  identification  is  clearly  fundamental 
for  application  of  control  theory,  especially  the  theory  developed  in  the 
last  fifteen  years  or  so.  Kalman  filtering  has  found  widespread  applica¬ 
tion  in  the  guidance  and  control  of  aerospace  vehicles ,  laser  tracking  and 
pointing  systems  and  missile  designs.  However,  very  few  applications  of 
non-linear  filtering  are  known.  The  extended  Kalman  filter  which  is  often 
used  in  practice  is  basically  not  understood  from  a  scientific  point  of 
view.  The  success  of  passive  tracking  algorithms  requires  advances  in  the 
state  of  the  art. 

The  control  of  future  aircraft,  possibly  statistically  unstable,  is  a 
problem  of  continuing  importance  from  the  point  of  view  of  designing 
adaptive  stability  augmentation  systems  that  operate  reliably  over  a  wide 
operating  envelope.  Similarly,  the  control  of  advanced  jet  engines,  whose 
dynamical  characteristics  change  rapidly  with  operating  conditions,  poses 
a  challenging  problem  if  one  wishes  to  design  a  control  system  which 
accomodates  commanded  thrust  level  changes  rapidly,  while  maintaining  fan 
and  compressor  stability  margins,  and  minimizing  excess  temperature  dura¬ 
tions.  Furthermore,  adaptive  control  appears  particularly  attractive  to 
handle  less  predictable  things  such  as  changes  in  atmospheric  gust 
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(particularly  wind  shears  on  final  approach)  and  changes  to  aircraft 
weight  and  mass  center  (e.g. ,  due  to  unusual  loading  distribution  or 
dropping  of  external  stores) . 

The  use  of  optical  sensors  and  laser  pointing  and  tracking  systems 
leads  to  new  problems  in  both  stochastic  estimation  and  control.  For 
example,  in  a  laser  tracking  system  the  "glint"  phenomenon  corresponds 
to  noise  in  the  measurement  equation  that  is  multiplicative  rather  than 
additive  in  nature.  Any  stochastic  interception  problem  involving  a 
maneuvering  target  and  with  "angle  only"  sensors  corresponds  to  a  very 
complex  nonlinear  estimation  and  control  problem. 

Our  research  progress  during  the  past  year  is  described  under  two 
main  headings.  Section  2  contains  a  discussion  of  progress  in  nonlinear 
filtering  and  stochastic  control.  Section  3  contains  a  discussion  of 
progress  on  adaptive  control  systems.  Section  4  is  a  list  of  references, 
while  Section  5  contains  a  list  of  publications  published  or  generated 
during  the  past  year,  supported  in  part  or  in  full  by  AFOSR  Grant  77-3281. 
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2.  NONLINEAR  FILTERING  AND  STOCHASTIC  CONTROL 
2.1  Nonlinear  Filtering 

Significant  progress  has  been  made  in  the  area  of  non-linear  filtering 
by  Professor  Mitter  and  his  graduate  students,  Daniel  Ocone  and  Larry  Vallot. 
This  work  has  been  reported  in  various  papers  and  two  theses  (References 
[1]  to  [10])  all  supported  by  this  grant.  The  progress  that  has  been 
made  is  on  several  fronts: 

(i)  The  innovations  problem  of  non-linear  filtering  in  the  form 
conjectured  by  Kailath  has  been  finally  settled  [1], 

(ii)  A  new  attack  on  the  non-linear  filtering  problem  based  on 
the  analysis  of  the  Zakai  equation,  proposed  independently 

by  Brockett  and  Mitter  has  had  considerable  success.  In  fact, 
a  major  part  of  the  summer  school  sponsored  by  NATO  in  Les 
Arcs,  France  held  in  July  1980  was  devoted  to  ideas  proposed 
in  [2] . 

(iii)  A  rigorous  formulation  of  the  variational  principle  under¬ 
lying  non-linear  filtering  has  been  presented  in  [3] .  This 
gives  new  insight  into  questions  of  approximation  for  non¬ 
linear  filters. 

(iv)  It  has  been  shown  that  there  are  indeed  deep  connections 
between  non-linear  filtering  and  recent  ideas  of  quantum 
physics  from  a  stochastic  viewpoint  ([2],  [3]). 

(v)  In  [8]  it  has  been  shown  how  these  recent  developments 

can  be  used  to  obtain  sub-optimal  filters  (with  guaranteed 
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performance  improvement)  for  bilinear  stochastic  systems. 

Previous  work  on  non-linear  filtering  has  concentrated  on  the  equation 
for  the  conditional  density  of  the  state  given  the  observations  over  a  time 
interval.  This  equation,  the  so-called  Kushner-Stratanovich  equation,  is 
a  non-linear  stochastic  partial  differential  equation  with  an  integral 
term.  In  contrast,  the  recent  work  of  Mitter  has  concentrated  on  the 
equation  for  the  unnormalized  conditional  density,  the  so-called  Duncan- 
Mortansen-Zakai  equation.  This  is  a  bilinear  stochastic  partial  dif¬ 
ferential  equation  which  has  as  its  input  the  observation  process  and, 
hence,  is  considerably  simpler  to  analyze  than  the  Kushner-Stratanovich 
equation.  This  equation  acts  as  a  -universal  recursive  filter,  in  the 
sense  that  any  conditional  statistic  can  be  computed  by  an  integration 
with  respect  to  this  unnormalized  density  and  an  appropriate  normalization. 

A  general  program  for  analyzing  this  equation  and  its  invariance 

properties  was  proposed  in  [2] .  In  this  program  a  certain  Lie  algebra 

1  2 

of  operators  with  two  generators  L*  ”  ~2  L1  ant3-  Li  w^ere  lq  i-s  the  formal 
adjoint  of  the  generator  of  a  diffusion  process  and  L^  is  a  multiplication 
operator  has  an  important  role  to  play.  If  this  Lie  algebra  is  finite 
dimensional  then  it  is  an  indication  that  the  Zakai  equation  has  a  finite¬ 
dimensional  statistic.  In  fact,  the  n-dimensional  Kalman  filtering  problem 
gives  rise  to  a  (2n+2) -dimensional  Lie  algebra  which  is  an  n-dimensional 
generalization  of  the  Oscillator  algebra  of  some  fame  in  quantum  physics. 
Indeed  the  Kalman  filter  occupies  the  same  role  as  the  Harmonic  oscillator 
or  the  free  field  in  quantum  physics  ([2],  [3]).  A  by-product  of  this 
approach  is  that  the  Zakai  equation  for  the  Kalman  filter  has  a  finite- 
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dimensional  statistic  even  if  the  initial  density  is  non-guassian.  Soon 
after  these  ideas  became  known  Benes  exhibited  finite-dimensional  filters 
for  a  class  of  filtering  problems  with  non-linear  drift  and  it  was  shown 
by  Mitter  [2]  that  the  filtering  problems  of  Benes  were  "gauge  equivalent" 
to  the  Kalman  filter.  In  the  doctoral  dissertation  of  D.  Ocone  [7]  it  is 
proved  that  for  scalar  diffusion  processes  defined  over  the  whole  real 
line ,  the  Kalman  filtering  problem  and  the  gauge  equivalent  Benes  filtering 
problems  were  the  only  problems  admitting  finite-dimensional  filters. 

In  [2]  it  was  shown  that  the  Lie  algebra  of  operators  corresponding 
to  a  large  class  of  filtering  problems  was  infinite-dimensional  and  simple. 
In  particular,  it  was  shown  that  the  Lie  algebra  of  operators  corresponding 
to  the  cubic  sensor  problem  (extensively  investigated  by  R.S.  Bucy)  was 
the  Weyl  algebra  (the  Lie  algebra  of  partial  differential  operators 
of  all  order)  which  is  known  to  be  infinite-dimensional  and  simple.  This 
result  was  obtained  independently  by  M.  Hazewinkel  and  S.  Marcus.  It  was 
conjectured  in  [2]  that  no  finite-dimensionally  computable  statistic 
existed  for  this  problem.  This  has  now  been  proved  by  H.  Sussman. 

In  proving  the  above  result,  one  constructs  a  proof  of  the  Ansatz  of 
R.W.  Brockett,  namely,  that  if  there  exists  a  finite-dimensional  filter 
than  there  must  exist  a  certain  Lie  Algebra  homomorphism  between  the  Lie 
Algebra  of  operators  corresponding  to  the  filtering  problem  and  the  Lie 
algebra  of  certain  vector  fields.  For  the  cubic  sensor  problem  this  was 
done  by  M.  Haxewinkel  and  S.  Marcus.  To  prove  the  theorem  on  the  non¬ 
existence  of  finite-dimensionally  computable  statistic  one  needs  a  rigorous 
proof  of  the  existence  and  uniqueness  of  solutions  of  the  Zakai  equation. 

In  joint  work  of  Mitter  with  J.S.  Baras  and  D.  Ocone  (preliminary  report 
in  [9])  this  has  been  accomplished  when  the  observation  map  is  a  poly¬ 


nomial. 
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In  D.  Ocone's  doctoral  thesis  [7],  the  problem  of  expanding  the  un¬ 
normalized  conditional  density  in  a  Ito-Wiener  Series  has  been  solved. 

Ocone  also  obtains  a  multiplication  formula  for  multiplying  two  multiple 
Ito  integrals  and  subsequently  expanding  it  in  an  Ito-Wiener  Series.  He 
has  also  obtained  integral  equations  for  the  kernels  corresponding  to  best 
polynomial  filters  and  shown  existence  and  uniqueness  of  solutions. 

In  L.  Vallot's  S.M.  thesis  [8],  a  theoretical  and  experimental  investi¬ 
gation  of  filtering  for  bilinear  systems  has  been  undertaken.  As  examples 
he  has  considered  the  phase-lock  loop  problem  and  also  the  cubic  sensor 
problem.  He  has  shown  how  tensor  and  Fock  space  ideas  expressed  in  [2] 
can  be  usefully  utilized  to  obtained  sub-optimal  filters  which  guarantee 
performance  improvement  (in  the  mean  square  sense) . 

In  summary,  a  framework  has  been  established  for  the  theoretical  and 
experimental  investigation  of  a  broad  class  of  non-linear  filtering  problems. 

It  is  believed  that  these  ideas  can  be  used  to  design  practical  non-linear 
filters  and  evaluate  their  performance. 

2.2  On  the  Relationship  between  Non-Linear  Filtering  and  Stochastic  Control 

It  has  long  been  felt  that  there  is  a  variational  principle  underlying 
non-linear  filtering.  Indeed  in  the  early  sixties,  a  formulation  of  the 
non-linear  filtering  problem  as  a  non-linear  least-squares  problem  was 
proposed  first  by  Bryson  and  Frazier  and,  subsequently,  by  Mortensen. 

These  formulations  were,  however,  deterministic  formulations  and  had  the 
inherent  mathematical  difficulty  of  working  with  white  noise,  the  formal 
differential  of  the  Wiener  process  (which  is  almost  surely,  not  differentiable) . 
It  was  also  known  that  these  formulations  gave  the  correct  answer  for  the 
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Kalman  Filtering  problem. 

In  [3] ,  Professor  Mitter  (joint  work  with  Professor  Fleming  of  Brown 
University)  has  shown  that  associated  with  the  Zakai  equation  of  non-linear 
filtering  is  a  Bellman-Hamilton- Jacobi  equation  which  corresponds  to  a 
perfectly  observable  stochastic  control  problem.  The  solution  of  one 
equation  leads  to  a  solution  of  the  other.  In  particular,  the  Kalman 
filtering  problem  corresponds  to  a  perfectly  observable  linear  quadratic 
gaussian  stochastic  control  problem.  Indeed,  this  explains  in  the 
clearest  possible  way  the  duality  principle  originally  proposed  by  Kalman. 
The  difficulty  of  the  non-differentiability  of  the  Wiener  paths  is  avoided 
by  working  with  the  robust  form  of  the  non-linear  filtering  equations  as 
developed  by  Clark  and  Davis  (M.H.A.  Davis  of  Imperial  College  was  a  visitor 
at  M.I.T.  in  1979  and  was  partially  supported  by  this  grant) . 

These  ideas  have  important  consequences.  It  opens  the  way  for  develop¬ 
ing  algorithms  for  non-linear  filtering  using  stochastic  control  ideas. 

It,  for  the  first  time,  explains  how  the  extended  Kalman  filtering  algorithm 
really  works.  Finally,  it  has  important  implications  in  the  development 
of  a  theory  of  stochastic  dissipativeness. 

2.3  Concluding  Remarks 

The  developments  in  non-linear  filtering  reported  here  have  many 
consequences.  For  example,  parameter  identification  problems  can  be 
considered  to  be  special  cases  of  non-linear  filtering  problems  and  the 
ideas  discussed  here  will  be  important  in  obtaining  a  better  understanding 
of  identification.  It  is  also  felt  that  the  relationship  between  non¬ 
linear  filtering  and  stochastic  control  exhibited  here  will  play  a  role 
in  obtaining  a  better  understanding  of  stochastic  adaptive  control. 
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3.  ADAPTIVE  CONTROL 

3.1  Introduction 

Significant  breakthroughs  in  our  understanding  of  adaptive  control 
systems  have  taken  place  during  the  past  year.  The  work  was  carried  out 
by  Dr.  L.  Valavani,  Prof.  Athans  and  Mr.  C.  Rohrs,  a  doctoral  student. 
Partial  results  are  documented  in  Refs.  [11] ,  [16] ,  and  [18] . 

During  the  past  year  theoretical  results,  backed  up  by  digital  computer 
simulation  studies,  that  relate  to  the  performance  of  several  adaptive 
control  algorithms  that  are  based  on  model  reference  techniques  were 
studied.  Special  emphasis  was  placed  upon  the  transient  performance  of 
these  algorithms  and  the  implications  upon  the  bandwidth  of  the  closed- 
loop  adaptive  system.  The  conclusions  are  as  follows: 

(1)  During  the  transient  phase  of  the  adaptation  procedure,  the 
control  signal  is  characterized  by  excessive  high  frequency 
content.  Novel  analytical  studies  based  upon  stability 
theory  can  be  used  to  predict  this  high  frequency  oscillatory 
behavior,  which  depends  both  upon  the  amplitude  and  frequency 
of  the  reference  input. 

(2)  During  the  transient  adaptation  phase  the  excessive  control  loop 
bandwidth  is  detrimental  to  system  performance,  because  it  can 
excite  unmodeled  high-frequency  dynamics  and  lead  to  instability. 

(3)  Similar  effects  can  occur  in  the  presence  of  stochastic  inputs 
(plant  noise  and  measurement  noise) . 

The  analytical  techniques  that  have  been  developed  are  constructive,  so 
that  modifications  to  the  existing  algorithms  are  suggested  to  overcome 
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the  practical  shortcomings  of  the  existing  algorithms. 

3.2  Background 

The  development  of  a  systematic  design  methodology  for  the  synthesis 
of  practical  self-adjusting  systems  which  can  maintain  first  stability 
and  second  performance  improvement  in  the  presence  of  rapid  and  large 
variations  in  the  open-loop  dynamics ,  represents  a  very  important  generic 
goal  in  control  system  engineering,  in  view  of  its  wide  applicability  to 
industrial  and  defense  applications.  The  so-called  "adaptive  control 
problem"  has  received  attention  by  theoreticions  and  practitioners  alike 
for  the  past  25  years.  About  a  dozen  books  and  hundreds  of  articles  have 
been  devoted  to  the  subject;  different  philosophies  have  been  developed 
(model  reference  adaptive  control,  self-tuning  regulators,  dual  control 
methods,  multiple-model  adaptive  control  etc.)  and  a  variety  of  (mostly 
academic)  examples  have  been  simulated. 

In  the  last  few  years  the  theory  of  adaptive  control  has  matured 
immensely.  Diverse  adaptive  algorithms  have  been  unified  under  the  same 
underlying  principles  (11]  and  global  asymptotic  stability  proofs  have 
been  obtained  for  a  large  class  of  such  algorithms  [12,  13,  14].  In  spite 
of  the  intense  research  activity,  however,  a  significant  gap  exists  between 
the  available  methodologies  and  the  potential  applications. 

As  a  rule,  these  algorithms  exhibit  rather  poor  behavior  in  practice. 
More  specifically,  although  certain  versions  of  the  self-tuning  regulator 
have  been  shown  to  work  on  systems  with  slow  dynamics,  such  as  ocean  tankers 
and  some  industrial  processes,  in  general  serious  problems  can  arise  when 
most  of  the  existing  algorithms  are  applied  to  systems  characterized  by 
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fast  time -constants ,  oscillatory  or  unstable  dynamics  and/or  unmodeled  high 
frequency  dynamics,  and/or  signficant  stochastic  disturbances  and  noisy 
sensor  measurements.  Moreover,  all  available  algorithms  can  exhibit 
transient  undesirable  behavior  which  makes  them  dangerous  to  use  in 
practical  applications,  e.g.  control  of  aircraft  [15],  For  the  most  part, 
the  difficulty  in  applying  theoretical  results  to  real  problems,  with 
reasonable  margins  of  confidence,  can  be  attributed  to  the  fact  that  the 
hypotheses  needed  to  rigorously  prove  the  theory  are  too  restrictive  from 
a  practical  point  of  view.  On  the  other  hand,  by  adopting  a  more  engi¬ 
neering  oriented  approach  to  the  formulation  of  adaptive  control  theories , 
we  have  a  better  chance  of  understanding  the  key  issues  on  which  to  base 
design  improvements.  A  systematic  study  and  analysis  of  the  convergence 
properties  of  already  existing  adaptive  algorithms  is  the  first  step  in 
this  direction. 

3.3  Research  Progress 

3.3.1  Motivation,  Focus,  and  Summary 

The  research  carried  out  in  the  last  year  was  motivated  mainly  from 
the  considerations  above.  In  the  process,  it  became  obvious  that  the 
global  black  box  approach  should  be  abandoned  and  that  frequency  domain 
methods  should  be  incoporated  with  time-domain  adjustment  mechanisms. 

Besides,  so  far,  preliminary  (limited)  experience  with  most  adaptive  schemes 
had  shown  that  they  tended  to  exhibit  undesirable  high-frequency  (behavior 
characteristics)  in  the  early  stages  of  adaptation  .  In  addition,  for 
practical  and  reliable  design,  one  should  always  look  for  good  approxima¬ 
tions  in  the  frequency-domain  with  due  consideration  for  cross-over  frequency 
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and  high  frequency  roll-off.  However,  in  the  case  of  the  present  adaptive 
schemes,  roll-off  circuits  (at  the  plant  input)  cannot  be  introduced 
without  violating  the  relative  degree  assumption  which  is  critical 
in  their  stability  analysis.  Such  a  modification  would  change  the  over¬ 
all  structure  of  adaptive  design  so  far.  On  the  other  hand,  future  "good" 
algorithms  should  also  be  able  to  handle  plants  with  oscillatory  or 
unstable  poles,  lots  of  zeros,  unmodeled  dynamics,  sensor  noise  and 
other  disturbances  during  the  entire  adaptation  process.  Therefore, 
special  emphasis  was  placed  upon  the  transient  performance  of  the 
adaptive  algorithms,  during  which  most  of  them  exhibit  rather  poor 
behavior  and  on  the  implications  upon  the  bandwidth  of  the  closed- loop 
adaptive  system.  Detailed  digital  computer  simulation  studies  of 
several  adaptive  algorithms  -  primarily  based  on  model  reference  techniques 
were  carried  out  during  the  past  year;  the  conclusions  of  this  work  can 
be  summarized  as  follows: 

(1)  During  the  transient  phase  of  the  adaptation  procedure, 
the  control  signal  is  characterized  by  excessive  high 
frequency  content.  Novel  analytical  studies  based  upon 
stability  theory  can  now  be  used  to  predict  this  high 
frequency  behavior,  which  depends  both  upon  the  amplitude 
and  the  frequency  of  the  reference  input. 

(2)  During  the  transient  adaptation  phase  the  excessive 
control  loop  bandwidth  is  detrimental  to  system  per¬ 
formance,  because  it  can  excite  unmodeled  high-frequency 
dynamics  and  lead  to  instability. 
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(3)  Similar  effects  can  occur  in  the  presence  of  stochastic 

inputs  (plant  noise  and  measurement  noise) .  The  analytical 
techniques  that  have  been  developed  are  constructive,  so 
that  modifications  to  the  existing  algorithms  are  suggested 
to  overcome  the  practical  shortcomings  of  the  existing 
algorithms . 

3.3.2  Discussion  of  Results 

The  first  part  of  our  research  is  described  in  more  detail  in  [16] . 
There,  results  of  digital  simulations  are  reported  for  cases  of  accurate 
modeling,  modeling  errors  which  exclude  high  frequency  dynamics  and  with 
observation  noise  present  at  the  output.  It  was  observed  that,  even  when 
the  system  was  properly  modeled,  the  plant  was  subjected  by  the  control 
input  to  high  frequencies  of  considerable  magnitude,  during  the  early 
stages  of  adaptation.  This  generation  of  high-frequency  control  inputs 
is  an  inherent  feature  of  current  adaptive  algorithms  due  to  a  multi¬ 
plicative  effect  both  in  the  parameter  adjustment  law  (i.e.,  eqns.2c)  , 

2d  following)  as  well  as  in  the  control  input  synthesis  (i.e.,  eqn.  lc) , 
since  time-varying  signals  are  used  in  both  the  above  procedures.  Clearly, 
such  high-frequency  inputs  are  undesirable ,  as  they  will  inevitably  excite 
unmodeled  dynamics  that  can  in  turn  cause  instability. 

Secondly,  it  was  also  shown  [16]  that,  in  the  presence  of  an  un¬ 
modeled  high-frequency  pole,  the  adaptive  system  exhibited  wildly  os¬ 
cillatory,  and  even  unstable,  closed  loop  behavior. 

Lastly,  the  presence  of  even  a  small  amount  of  observation  noise 
prevented  the  closed  loop  system  from  converging  to  the  desired  model 
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by  causing  it  to  slowly  drift  away  to  an  increasingly  higher  bandwidth 
system.  As  a  consequence,  high  frequency  unmodeled  dynamics  will  eventu¬ 
ally  be  excited  even  at  an  advanced  stage  of  the  adaptation  process. 

This  increase  in  the  bandwidth  of  the  closed- loop  system  was  also 
observed  by  Narendra  and  Peterson  [17]  in  terms  of  a  scalar  example. 

For  the  second  part  of  our  work  reported  in  [18] ,  in  order  to  ob¬ 
tain  practical  insights  as  well  as  analytical  results,  a  simple  scalar 
system  was  first  studied  in  a  systematic  way.  The  equations  that  de¬ 
scribe  it  are  given  below: 

Plant  Dynamics:  y(t)  =  -ay(t)  +  $u(t)  (l.a) 

» 

Model  Dynamics:  y  (t)  =  -ay  (t)  +  br(t)  (l.b) 

m  m 

where  the  plant  control  is  given  by  u(t)  =  0^(t)y(t)  +  02(t)r(t)  ,  (l.c) 

with  r(t)  being  the  reference  input  and  9^(t),  ©2(t)  adjustable  (control) 

parameters  used  in  the  control  signal  synthesis.  The  first  algorithm  for 
the  control  parameter  adaptation  studied  was  that  proposed  by  Narendra 
and  Valavani  [19] ;  its  scalar  version  consists  of  the  following  system  of 
nonlinear  differential  equations  for  the  output  error  e(t)  and  parameter 
errors  <f>^(t),  <}>2(t)  as  well  as  the  plant  state  y(t)  : 

fit)  =  -ay(t)  +  gu(t)  (2.  a) 

e(t)  =  -ae  (t)  +  bc^tJyCt)  +  b<|>2(t)r(t)  (2.b) 

^(t)  =  -yi;]y(t)e(t)  -  y12e(t)r(t)  (2.c) 

$2(t)  =  Y21Y(t)e(t)  -  Y22e (t) r (t) 


(2.d) 
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where  e(t)  =  y  (t)  -  y(t)  and  (J>^(t),  4>2  (t)  are  the  parameter  errors; 

Yij /  ifj  =  1,2,  are  constants  and  such  that  Yi±  >  0,  Yi;lY22  >  ^12Y21‘ 

A  stability-based  time-varying  analysis  of  the  properties  of  the  non¬ 
linear  time-varying  system  described  by  eqns.  (2)  results  in  the  develop¬ 
ment  of  a  new  analytical  method  that  explains  and  even  predicts  the 
behavior  of  the  adaptive  system  observed  in  the  simulations.  The 
analytical  results  thus  obtained  are  valid  in  the  vector  case  as  well; 
they  also  provide  further  insights  into  the  mechanism  of  adaptation.  Due 
to  the  inherent  nonlinearity  of  the  closed- loop,  it  was  found  that  the 
behavior  of  the  system  is  sensitive  to  both  the  magnitude  of  the  reference 
input  and  the  input  frequencies.  A  large  reference  input  magnitude  can 
create  arbitrarily  high  frequency  content  in  the  adjustable  (control) 
parameters  ©^(t)  and  (t)  and,  therefore,  also  in  the  control  signal 
u(t)  . 

Moreover  this  analysis  demonstrates  that,  cj)^ (t)  and  cj)^  (t)  ,  do  not 
necessarily  converge  to  zero  but,  rather,  to  a  linear  subspace  in  the 
parameter  space  where  the  output  error  is  identically  zero.  This  sub¬ 
space  evolves  in  time,  and  its  evolution  is  dependent  on  the  time-varying 
characteristics  of  the  reference  input  and  the  plant  output.  If  the 
reference  input  is  such  that  the  subspace  remains  fixed,  no  changes  in  the 
parameters  occur.  If,  however,  the  subspace  varies  with  time,  the 
parameter  errors  will  approach  it  from  a  direction  orthogonal  to  it. 

It  is  precisely  here  that  the  so  called  "richness"  of  the  adaptive  signals 
could  play  an  important  role,  insofar  as  parameter  error  convergence 
is  concerned.  It  is  becoming  increasingly  apparent,  however,  that  the 
closed  loop,  given  the  present  algorithms,  cannot  satisfy  the  "richness" 
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conditions  required  for  asymptotic  convergence.  This  has  obvious  impli¬ 
cations  for  certain  adaptive  schemes  [20,  21]  whose  overall  global  stability 
hinges  heavily  on  exact  parameter  convergence.  On  the  other  hand,  stability 
results  have  been  obtained  recently  for  the  above  mentioned  class  of 
algorithms,  irrespective  of  "sufficient  excitation"  and  parameter  con¬ 
vergence  [18] ;  but  they  are  valid  only  locally. 

When  the  algorithm  described  in  [19]  was  studied  with  an  unmodeled 
high  frequency  pole,  the  same  analytical  results  successfully  explained 
the  exhibited  oscillatory,  and  eventually  unstable,  behavior  and,  more¬ 
over,  demonstrated  the  same  dependence  of  such  behavior  on  the  reference 
input . 

In  the  case  of  observation  noise,  n(t) ,  our  analysis  shows  that  a 

2 

term  containing  n  (t)  will  in  effect  keep  driving  the  error  system  and, 
therefore,  the  parameter  error  can  increase  within  the  zero  output  sub¬ 
space  discussed  above.  But  if  the  variance  of  the  measurement  noise  is 
known,  this  term  can  be  subtracted  out.  Unf ortunately ,  the  problem  still 
remains  of  white  noise  being  input  into  a  marginally  stable  system  in  the 
same  subspace.  However,  since  the  noise  also  drives  (affects)  the 
time-evolution  of  the  subspace ,  it  may  very  well  be  the  case  that  the 
decaying  effect  of  the  time  evolution  of  the  subspace  on  the  parameter 
errors  is  enough  to  keep  their  variance  bounded. 

The  analytical  framework  mentioned  in  the  foregoing  was  employed 
to  study  different  adaptive  algorithms  and  consistency  of  the  results 
was  shown.  Specifically,  our  study  so  far  has  concentrated  on  the  algorithms 
obtained  using  a  stability  point  of  view  for  their  derivation,  the  most 
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important  representatives  of  which  are  Monopoli's  [23]  Narendra,  Valavani , 
and  Lin's  [19,  12],  Feuer  and  Morse's  [24],  Laudau  and  coworkersr  [25],  both 
for  discrete  as  well  as  continuous-time  systems.  In  the  analysis,  the 
above  algorithms  all  shared  the  same  characteristics,  although  some  ex¬ 
hibited  marginally  better  behavior  in  some  numerical  cases.  For  example, 
the  algorithm  described  in  [12]  remained  stable,  in  the  presence  of  one 
unmodeled  pole,  with  increasing  reference  input  amplitude,  although  the 
control  signal  was  heavily  oscillatory.  However,  the  presence  of  two  un¬ 
modeled  high-frequency  poles  resulted  in  its  instability.  Conversely, 
the  observation  noise  enters  here  in  a  much  more  complicated  way  than 
in  the  rest  of  the  algorithms  examined. 

3.4  Future  Research  Plans 

A  class  of  adaptive  algorithms  which  are  currently  being  investi¬ 
gated  in  the  framework  established  in  [18] ,  is  that  containing  Astrom 
and  Wittenmark's  basic  self-tuning  regulator  scheme  [26]  and  those  that 
followed  it  along  the  same  lines;  i.e.,  some  schemes  contained  in  [27], 
and  the  algorithms  obtained  by  Goodwin,  Ramadge  and  Caines  [13] .  These 
are  basically  dead-beat  algorithms  designed  for  discrete-time  systems. 
Although  at  first  glance  they  seem  to  have  some  advantage  over  the  others 
with  respect  to  the  high  frequency  content  of  the  control  input,  we 
anticipate  a  much  worse  response  in  the  presence  of  unmodeled  dynamics. 

Work  is  currently  in  progress  to  verify  this  preliminary  conjecture. 

Based  on  our  theoretical  results,  we  are  also  developing  modifi¬ 
cations  to  the  existing  schemes  that  avoid  the  undesirable  behavior  de¬ 
scribed  so  far.  The  full  paper  [18]  will  describe  some  of  the  results 
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current  ly  obtained. 

Finally,  apart  from  the  modifications  and  improvements  for  determi¬ 
nistic  adaptive  algorithms,  the  effect  of  process  and/or  measurement 
noise  has  to  be  expressly  taken  into  account  in  all  design  improvements 
and  the  robustness  properties  of  the  resulting  adaptive  schemes  have 
to  be  analyzed.  A  lot  of  future  research  is  required  along  these  lines, 
before  adaptive  algorithms  can  have  practical  impact,  apart  from  the 
purely  theoretical  appeal  they  have  enjoyed  so  far. 
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